A candidate claims that quantization always speeds up search. Under what conditions might quantization with rescoring actually be slower than searching un-quantized vectors?

Quantization speeds up the traversal by using smaller vectors, which reduces memory bandwidth and cache misses. But when the configuration demands very high recall, the rescoring step can dominate the total cost, and the overall search can be slower than searching un-quantized vectors. The mechanism is that rescoring performs a full-precision distance computation for every candidate in the oversampled set. The oversampled set is limit * oversampling, and for high recall the oversampling factor may be 5, 10, or more. Each rescore is 4x to 32x more expensive than a quantized distance computation, depending on the scheme. So the rescoring cost is (limit * oversampling) * full_precision_cost, which can exceed the savings from the faster traversal. The second condition is the quantization scheme: binary quantization requires more oversampling than scalar, so its rescoring cost is higher. The third condition is the candidate set size: a reranker over a large candidate set, combined with rescoring, multiplies the full-precision work. The fourth condition is the hardware: if the collection is small enough to fit in cache, the un-quantized traversal is already fast, and the quantization overhead (decompression, rescoring) can make it slower. The fifth condition is the metric and the vector dimension: for low dimensions, the quantized distance computation is not much faster than the full-precision one, so the savings are small relative to the rescoring cost.

The mechanism that determines whether quantization is a net win is the ratio of the traversal cost to the rescoring cost. If the traversal dominates (large collection, many candidates, high ef), quantization is a clear win because it speeds up the dominant part. If the rescoring dominates (high oversampling, large candidate set, high recall target), quantization can be a net loss because the rescoring is full-precision and grows with the oversampled set. The break-even point depends on the oversampling factor, the vector dimension, the quantization scheme, and the hardware. In practice, the right approach is to measure the end-to-end latency with and without quantization at the same recall level, not just the traversal latency. The common mistake is to measure the quantized traversal in isolation and conclude that quantization is faster, without accounting for the rescoring. The second mistake is to set a very high oversampling factor by default, which makes the rescoring dominate. The third mistake is to compare the quantized configuration at a higher recall than the unquantized one and attribute the latency difference to quantization when it is actually the recall target. Version note: the quantization search parameters (rescore, oversampling) and the exact behavior of rescoring have changed across Qdrant releases. The default oversampling and whether rescoring is on by default differ. Measure on your version at the recall level you plan to ship.

The trade-off is between memory savings and the rescoring cost. Quantization always saves memory; it does not always save latency. For a memory-constrained deployment, quantization is worth it even if the latency is the same, because it enables the collection to fit. For a latency-constrained deployment where the collection already fits in RAM, quantization may not be worth it. The common mistakes are: (1) assuming quantization always speeds up search; (2) measuring the traversal in isolation; (3) comparing configurations at different recall levels; (4) setting oversampling too high; (5) not considering the candidate set size for a downstream reranker. Version note: the quantization schemes, the oversampling parameter, and the rescoring behavior have evolved across Qdrant releases. Measure on your version with your data.

Version-dependent: the quantization search parameters and the rescoring behavior have changed across Qdrant releases. The default oversampling and whether rescoring is on by default differ. Measure on your version at the recall level you plan to ship.